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Abstract — We propose a novel decomposition framework for 
the distributed optimization of general nonconvex sum-utility 
functions arising naturally in the system design of wireless multi- 
user interfering systems. Our main contributions are: i) the 
development of the first class of (inexact) Jacobi best-response 
algorithms with provable convergence, where all the users simul- 
taneously and iteratively solve a suitably convexified version of 
the original sum-utility optimization problem; ii) the derivation 
of a general dynamic pricing mechanism that provides a unified 
view of existing pricing schemes that are based, instead, on 
heuristics; and iii) a framework that can be easily particularized 
to well-known applications, giving rise to very efficient practical 
(Jacobi or Gauss-Seidel) algorithms that outperform existing ad- 
hoc methods proposed for very specific problems. Interestingly, 
our framework contains as special cases well-known gradient 
algorithms for nonconvex sum-utility problems, and many block- 
coordinate descent schemes for convex functions. 

I. Introduction 

WIRELESS networks are composed of users that may 
have different objectives and generate interference, 
when no multiplexing scheme is imposed a priori to regulate 
the transmissions; examples are peer-to-peer, ad-hoc, and cog- 
nitive radio systems. A usual and convenient way of designing 
such multiuser systems is by optimizing the "social function", 
i.e., the (weighted) sum of the users' objective functions. 
Since centralized solution methods are too demanding in most 
applications, the main difficulty of this formulation lies in 
performing the optimization in a distributed manner with lim- 
ited signaling among the users. When the social problem is a 
sum-separable convex programming, many distributed methods 
have been proposed, based on primal and dual decomposition 
techniques; see, e.g., ||2)-||4) and references therein. In this 
paper we address the more frequent and difficult case in which 
the social function is nonconvex. It is well known that the 
problem of finding a global minimum of the social function is, 
in general, NP hard (see e.g. (5)), an d even centtalized solution 
methods are in jeopardy. As a consequence, most current 
research efforts have been focused on finding efficiently high 
quality suboptimal solutions via low complexity (possibly) 
distributed algorithms. A recent survey on nonconvex resource 
allocation problems in interfering networks modeled as Gaus- 
sian Interference Channels (ICs) is J6j. 

In an effort to obtain distributed albeit suboptimal algo- 
rithms a whole spectrum of approaches has been explored, 
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trying to balance practical effectiveness and coordination 
requirements. At one end of the spectrum we find game- 
theoretical approaches, where users in the network are mod- 
eled as players that greedily optimize their own objective 
function. Game-theoretical models for power control prob- 
lems over ICs have been proposed in iTTI- lfTTI and lfl2l - 
Ifl4l for SISO and MISO/MIMO systems, respectively. Two 
recent tutorials on the subject are lfj"5l . Ifl6l . while recent 
contributions using the more general mathematical theory of 
Variational Inequalities ifTTl are |[T8l - ll20l . The advantage 
of game-theoretic methods is that they lead to distributed 
implementations (only local channel information is required 
at each user); however they converge to Nash equilibria that 
in general are not even stationary solutions of the nonconvex 
social problem. In contrast, other methods aim at reaching 
stationary solutions of the nonconvex social problem, at the 
cost of more signaling and coordination. Sequential decom- 
position algorithms were proposed in fl2Tl - ll24l for the sum- 
rate maximization problem over SISO/MIMO ICs, and in ll25l 
for more general (nonconvex) functions. In these algorithms, 
only one agent at a time is allowed to update his optimization 
variables; a fact that in large scale networks may lead to 
excessive communication overhead and slow convergence. 

The aim of this paper is instead the study of more appealing 
simultaneous distributed methods for general nonconvex sum- 
utility problems, where all users can update their variables at 
the same time. The design of such algorithms with provable 
convergence is much more difficult, as also witnessed by 
the scarcity of results available in the literature. Besides the 
application of the the classical gradient projection algorithm to 
the sum-rate maximization problem over MIMO ICs 11261 . par- 
allel iterative algorithms (with message passing) for DSL/ad- 
hoc SISO networks and MIMO broadcast interfering channels 
were proposed in ll27l - ll29l and ll30l . respectively. Unfortu- 
nately, the gradient schemes |26l suffer from slow convergence 
and do not exploit any degree of convexity that might be 
present in the sum-utility function; l27l - ll29l hinge crucially 
on the special log-structure of the users' rate functions; and 
l30l is based on the connection with a weighted MMSE 
problem. This makes Il27l - ll30l not applicable to different 
classes of sum-utility problems. 

Building on the idea first introduced in JT], the main 
contribution of this paper is to propose a new decomposition 
method that: i) converges to stationary points of a large class of 
(nonconvex) social problems, encompassing most sum-utility 
functions of practical interest (including functions of complex 
variables); ii) decomposes well across the users, resulting in 
the parallel solution of convex subproblems, one for each user; 
iii) converges also if the users' subproblems are solved in an 
inexact way; and iv) contains as special case the gradient algo- 
rithms for nonconvex sum-utility problems, and many block- 
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coordinate descent schemes for convex functions. Moreover, 
the proposed framework can be easily particularized to well- 
known applications, such as l2ll - ll24"l . (29), 1311 , giving rise 
in a unified fashion to distributed simultaneous algorithms that 
outperform existing ad-hoc methods both theoretically and 
numerically. We remark that while we follow the seminal ideas 
put forward in [T), m this paper, besides providing full proofs 
of the results in fT), we i) consider a much wider class of 
social-problems and (possibly inexact) algorithms, including 
U as special cases, ii) discuss in detail the case of functions 
of complex variables, and iii) compare numerically to state-of- 
the-art alternative methods. To the best of our knowledge, this 
paper is the first attempt toward the development of decompo- 
sition techniques for general nonconvex sum-utility problems 
that allow distributed simultaneous (possibly inexact) best- 
response-based updates among the users. 

On the one hand, our approach draws on the Successive 
Convex Approximation (SCA) paradigm, but relaxes the key 
requirement that the convex approximation must be a tight 
global upper bound of the social function, as required instead 
in 11271 . 11321 . If33l (see Sec fVT] for a detailed comparison with 
ll27l . 11321 . 1331 ). This represents a turning point in the design 
of distributed SCA-based methods, since up to date, finding 
such an upper bound convex approximation for sum-utility 
functions having no specific structure (as, e.g., ||241 . 11261 - 1301 ) 
has been an elusive task. 

On the other hand, our method also sheds new light on 
widely used pricing mechanisms: indeed, our scheme can be 
viewed as a dynamic pricing algorithm where the pricing rule 
derives from a deep understanding of the problem characteris- 
tics and is not obtained on an ad-hoc basis, as instead in [21 j- 
lf24l . II3T1 . We conclude this review by mentioning the recent 
work (34), where the authors, developing ideas contained in 
ll30l . 1331 . proposed parallel schemes based of the SCA idea 
that are applicable (only) to the class of sum-utility problems 
for which a connection with a MMSE formulation can be 
established. Note that 11331 1134*1 . which share some ideas with 
our approach, appeared after JT]. 

The rest of the paper is organized as follows. Sec. [U] 
introduces the sum-utility optimization problem along with 
some motivating examples. Sec. [Ill] presents our novel de- 
composition mechanism based on partial linearizations; the 
algorithmic framework is described in Sec. [fV] Sec. ""/"extends 
our results to sum-utility problems in the complex domain; 
further generalizations are discussed in Sec. [VI] In Sec. IVIII 
we apply our new algorithms to some resource allocation 
problems over SISO and MIMO ICs, and compare their 
performance with the state-of-the-art decomposition schemes. 
Finally, Sec. I Villi draws some conclusions. 

II. Problem Formulation 

We consider the design of a multiuser system composed of / 
coupled users I = {1, . . . , I}. Each user i makes decisions on 
his own rii -dimensional real strategy vector x;, which belongs 
to the feasible set JCf, the vector variables of the other users 
is denoted by x_; = (xj)j^.; e JC-i = Wj^JCf, me users' 
strategy profile is x= (xi)|_ 1; and the joint strategy set of the 
users is JC = Yijei^j- The system design is formulated as 



minimize U(x) = > iH x ) 

(1) 

subject to Xi 6 JCi, Vi E 1, 
with l f = {1,...,//}. Ob serve that, in principle, the set If 
of objective functions is different from the set I of users; we 
show shortly how to explore this extra degree of freedom to 
good effect. Of course, ([T]) contains the most common case 
where there is exactly one function for each user, i.e. I = If. 
Assumptions. We make the following blanket assumptions: 
Al) Each K-i is closed and convex; 
A2) Each fi is continuously differentiable on JC; 
A3) VU is Lipschitz continuous on JC, with constant Lsju\ 
A4) The lower level set £(x°) = {x e JC : U(x) < U{x )} 
of the social function U is compact for some x° € IC. 

The assumptions above are quite standard and are satisfied 
by a large class of problems of practical interest. In particular, 
condition A4 guarantees that the social problem has a solution, 
even when the feasible JC is not bounded; if IC is bounded A4 is 
trivially satisfied. A sufficient condition for A4 when JC is not 
necessarily bounded is that U be coercive [i.e., C/(x) — > +oo 
as ||x|| — > +oo, with x € JC]. Note that, differently from 
classical Network Utility Maximization (NUM) problems, here 
we do not assume any convexity of the functions fi, which 
leads to a nonconvex optimization problem ([T). For the sake 
of simplicity, in (Q"]i we assume that the users' strategies are 
real vectors; in Sec. [Vj we extend our framework to complex 
matrix strategies, to cover also the design of MIMO systems. 
A motivating example. The social problem (["Qi is general 
enough to encompass many sum-utility problems of practical 
interest. It also includes well-known utility functions studied 
in the literature; an example is given next. Consider an N- 
parallel Gaussian IC composed of / active users, and let 
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be the maximum achievable rate on link i, where = 
(pik)k=i denotes the power allocation of user i over the N 
parallel channels, = (pj)j^i is the power profile of all 
the other users j 7^ i, \H l} (fc)T is the gain of the channel 
between the j-th transmitter and the z-th receiver, af fc is the 
variance of the thermal noise over carrier fc at the receiver i, 
and \Hij (fc)| 2 pjk represents the multiuser interference 

generated by the users j ' ^ i at the receiver i. Each transmitter 
i is subject to the power constraints p; G Vi, with 

Pi — {pi € : W iPi < I.r x } , (2) 

where the inequality, with given l™ ax £ R" li and W, E 
R" iiXjv has to be intended component- wise. Note that the 
linear (vector) constraints in (f2) are general enough to model 
classical power budget constraints and different interference 
constraints, such as spectral mask or interference-temperature 
limits. Finally, let fi : M + — > M be the utility functions of the 
users' rates. The system design can then be formulated as 



maximize V" fi (n(pi, p_ 4 )) 
pi,...,pj *—± 

subject to p.; G Pi, Vi gl. 



(3) 



3 



Note that (0 is an instance of (0, with If = I; moreover 
assumptions A1-A4 are satisfied if the utility functions f%{x) 
are i) concave and nondecreasing on K + , and ii) continuously 
differentiable with Lipschitz gradients. Interestingly, this class 
of functions fi(x) include many well-known special cases 
studied in the literature, such as the weighted sum-rate func- 
tion, the harmonic mean of the rates, the geometric mean of 
(one plus) the rates, etc.; see, e.g., J6), |2TI . Il22l . (35). 

Since the class of problems is in general nonconvex 
(generally NP hard Q), the focus of this paper is on the de- 
sign of distributed solution methods for computing stationary 
solutions (possibly local minima) of (0. Our major goal is to 
devise simultaneous best-response schemes fully decomposed 
across the users, meaning that all the users can solve in 
parallel a sequence of convex problems while converging to 
a stationary solution of the original nonconvex problem. 

III. A New Decomposition Technique 

We begin by introducing an informal description of our 
new algorithms that sheds light on the core idea of the 
novel decomposition technique and establishes the connection 
with classical descent gradient-based schemes. This will also 
explain why our scheme is expected to outperform current 
gradient methods. A formal descriptions of the proposed 
algorithms along with their main properties is given in Sec. 
[IV] for the real case, and in Sec. [V]for the complex case. 

A. Why do conditional gradient algorithms work poorly? 

A classical approach to solve a nonconvex problem like 
(0 would be using some well-known gradient-based descent 
scheme. A simple way to generate a (feasible) descent di- 
rection is for example using the conditional gradient method 
(also called Frank- Wolfe method) [4]: given the current iterate 
x" = (xf the next feasible vector x n+1 is given by 

x n+l = x n + d n (4) 

where d" = x' 1 - x™, x" = (x")f =1 is the solution of the 
following set of convex problems (one for each user): 

X? = argmin {v Xi C/ (x") T (x, - x?)} , (5) 

for all i E I, and 7™ <G (0, 1] is the step-size of the algorithm 
that needs to be properly chosen to guarantee convergence. 

Looking at (0 one infers that gradient methods are based on 
solving a sequence of parallel convex problems, one for each 
user, obtained by linearizing the whole utility function U(x) 
around x n , a fact that does not exploit any "nice" structure 
that the original problem may potentially have. 

At the basis of the proposed decomposition techniques, 
there is instead the attempt to properly exploit any degree 
of convexity that might be present in the social function. To 
capture this idea, for each user i G I, let Si C Xf be the set 
of indices of all the functions /j(xi,x_.;) that are convex in 
Xj on Id, for any given x_i G JC-C 

Si = {j G I/:/j(»,x_i)is convex on /C ? :,Vx_.; G /C_j} (6) 

and let Ci C Si be a given subset of Si. The idea is to preserve 
the convex structure of the functions in Ci while linearizing 



the rest. Note that we allow the possibility that Si = 0, even if 
we "hope" that Si ^ 0, and actually this latter case occurs in 
most of the applications of interest, see Sec. I VIII For each user 
i G I, we can introduce the following convex approximation 
of t/(x) around x" G JC: 

/ Ci (x i; x") 4 ^/ J -(x i ,x!l i ) + 7r Cj (x") T (x i -xr) 

+^(x i -x?) r H i (x")(x i -x?) (7) 

with 

where C_; = Z/\C« is the complement of Ci, Ti is a given 
positive constant, and H.;(x n ) is an rii x rii uniformly positive 
definite matrix (possibly dependent on x"), i.e. Hi(x") — 
CirT y 0, for some positive CH t . For notational simplicity, 
we omitted in fi (x^ ; x™ ) the dependence on r, and (x ra ) . 
Note that in (0, we add a proximal-like regularization term, 
in order to relax the convergence conditions of the resulting 
algorithm or enhance the convergence speed (cf. Sec. II V) . 

Associated with each /^(x^x™) we can define the follow- 
ing "best response" map that resembles (0: 

x Ci (x B , Ti) 4 argmin f Ci (x 4 ; x"). (9) 

XiG/Q 

Note that, in the setting above, xc^x",^) is always well- 
defined, since the optimization problem in (0 is strongly 
convex and thus has a unique solution. Given (0, we can 
introduce the best-response mapping of the users, defined as 

/C9y^x c (y,T)^(x Ci (y,T l ))Li- (10) 

The proposed search direction d" at point x™ in (0|i becomes 
then xc(x™,r) — x^™'. The challenging question now is 
whether such direction is still a descent direction for the 
function U at x" and how to choose the free parameters (such 
as Ti's, 7™'s, and Hi(y)'s) in order to guarantee convergence 
to a stationary solution of the original nonconvex sum-utility 
problem. These issues are addressed in the next sections. 

B. Properties of the best-response mapping Xc(y,r) 

Before introducing a formal description of the proposed 
algorithms, we derive next some key properties of the best- 
response map xc(y,r), which shed light on how to choose 
the free parameters in (0 and prove convergence. 

Proposition 1: Given the social problem (0 under Al)-A4), 
and t = (Tj)|_ 1 > and (Hi(x)) i=1 (such that each H;(x) — 
C//T h for all x G K, and some CH t > 0), the mapping 
JC 3 y h-> x(y, r) has the following properties: 

(a) xc(y,T) is Lipschitz continuous on JC, i.e., there exists a 
positive constant L such that 

||x c (y,x)-x c (z,r)|| < L ||y - z|| , Vy,zG/C; (11) 

(b) The set of the fixed-points of xc(y, t) coincides with the 
set of stationary solutions of the social problem ([Q; therefore 
xc(y, t) has a fixed-point; 



4 



(c) For every given y 6 JC, the vector xc(y, t) — y is a descent 
direction of the social function U (x) at y such that 

(xc(y,r) - y) T V x (7(y) < -c ||x c (y, r) - y|| 2 , (12) 

for some positive constant c > c T , with 

c T = min < t % ■ c Hz + min c/ c . (z_ ? ) > , (13) 

where c/ c (x_^) is the constant of strong convexity of 
EjGCi /j'(». x -i) on 

(d) If V x C/(x) is bounded on JC, then there exists a finite 
constant a > such that 

\\xc(y,T)-y\\ < a, Vy 6 JC. (14) 

Proof: See Appendix A. ■ 

Proposition Q] makes formal the idea introduced in Sec. 
IIII- Al and thus paves the way to the design of distributed best- 
response-like algorithms for (Q3 based on xc (y, r). Indeed, the 
inequality (O states that either (x c (x n ) - x' l ) T V x J7(x n ) < 
or Xc(x") = x". In the former case, d™ = xc(x") — x" 
is a descent direction of C/(x) at x"; in the latter case, x n is 
a fixed-point of the mapping xc(»,t) and thus a stationary 
solution of the original nonconvex problem ([T) [Prop. Q] (b)] . 

Quite interestingly, we can also provide a characterization of 
the fixed-points of xc(y, t) [and thus the stationary solutions 
of (Q~|)] in terms of Nash equilibria of a game with a proper 
pricing mechanism. Formally, we have the following. 

Proposition 2: Any fixed-point x* of xc(»,t) is a Nash 
equilibrium of the game where each user i £ 1 solves the 
following priced convex optimization problem: given x_j, 

min - y2fj( X i' X ~i) +7T Ci (x*) T Xi. (15) 

According to the above proposition, the stationary solutions 
of (Q~|) achievable as fixed-points of x^^r) are unilaterally 
optimal for the objective functions in (TTSb . This result is in 
agreement with those obtained in El . 11231 for the sum- 
rate maximization problem over SISO frequency selective- 
channels. Despite its theoretical interest, however, Prop.|2]does 
not help in practice to solve (fl}. Indeed, the computation 
of a Nash equilibrium of the game in (T5[ would require 
the a-priori knowledge of the prices 71-^ (x*) and thus the 
equilibrium itself, which of course is not available. 

IV. Distributed Decomposition Algorithms 

We are now ready to introduce our new algorithms, as a 
direct product of Prop. [TJ We first focus on (inexact) Jacobi 
schemes (cf. Sec. II V- Al l; then we show that the same results 
hold also for (inexact) Gauss-Seidel updates (cf. Sec. IIV-Q . 

A. Exact Jacobi best-response schemes 

The first algorithm we propose is a Jacobi scheme where 
all users update simultaneously their strategies based on the 
best-response x^^t) (possibly with a memory); the formal 
description is given in Algorithm Q] below, and its convergence 
properties are given in Theorem [3] 

'if X]j6C fj (•, x_i) is convex but not strongly convex, Cf c (x_j) = 0]. 



Algorithm 1: Exact Jacobi SCA Algorithm 

Data : r > 0, {7"} > 0, x° £ JC. Set n = 0. 
( S . 1 ) : If x™ satisfies a termination criterion: STOP; 
( S . 2 ) : For all i £ X, compute x Cl (x n , r) [cf. ©]; 
( S . 3 ) : Set x" +1 = x" + 7" (x (x n , r) - x™); 
( S . 4 ) : n <- n + 1, and go to ( S . 1 ) . 

Theorem 3: Given the social problem ([D under A1-A4, 
suppose that one of the two following conditions is satisfied: 

(a) For each i, H;(x) — c# ; I >z for all x £ JC and some 
c Hi > 0, and {7™} and r > are chosen so that 

< inf 7" < sup 7" < 7 max < 1 and 2c T > 7 max L V c/, 

" n 

1— 1 (16) 

with c T defined in ( Il3l l. 

(b) For each i, H;(x) — c#J >z for all x £ JC and some 
c Hi > 0, t > 0, and {7™} is chosen so that 

7 n £ (0, 1], 7™ -> 0, and ^7" = +00. (17) 

n 

Then, either Algorithm [1] converges in a finite number of 
iterations to a stationary solution of (fl} or every limit point 
of the sequence {x"}™ =1 (at least one such points exists) is 
a stationary solution of ([T} . Moreover, none of such points is 
a local maximum of U. 

Proof: See Appendix B. ■ 
Main features of Algorithm [T] The algorithm implements 
a novel distributed SCA decomposition: all the users solve in 
parallel a sequence of decoupled strongly convex optimization 
problems as in ([9). The algorithm is expected to perform better 
than classical gradient-based schemes (at least in terms of 
convergence speed) at the cost of no extra signaling, because 
the structure of the objective functions is better preserved. It 
is guaranteed to converge under very mild assumptions (the 
weakest available in the literature) while offering some flexi- 
bility in the choice of the free parameters [see conditions (a) 
or (b) of Theorem|3)- This degree of freedom can be exploited, 
e.g., to achieve the desired tradeoff between signaling and 
convergence speed, as discussed next. 

On the choice of the free parameters. Convergence of 
Algorithm[T]is guaranteed either using a constant step-size rule 
[cf. ( fTot l or a diminishing step-size rule [cf. (fTTTil. Moreover, 
different choices of {Ci} are in general feasible for a given 
social function, resulting in different best-response functions 
and signaling among the users. 

1) Constant step-size: In this case, 7™ = 7 < 7 max for 
all 71, where ^ max £ (0, 1] needs to be chosen together with 
t > and (Hi(y))f =1 so that the condition 2 c T > 7 max L V [/ 
is satisfied, with c T defined in ( fT3l . This can be done in 
several ways. An interesting special case worth mentioning 
is: 7 = 7 max = 1 for all n, H 4 (y) = I for all i £ I, 
and r > large enough so that 2 c T > L^u- This choice 
leads to the classical Jacobi best-response scheme (but with a 
proximal regularization), namely: at each iteration n, x" +1 = 
xci (x",t) , for all i £ I. To the best of our knowledge, 
this algorithm along with its convergence conditions [Theorem 
[3^)] represents a new result in the optimization literature; 
indeed classic best-response nonlinear Jacobi schemes require 
much stronger (sufficient) conditions to converge (implying 
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contraction) |4] Ch. 3.3.5]. Note that the choice of Tj's to 
guarantee convergence [i.e., 2c T > Lsjjj] can be done locally 
by each user with no signaling exchange, once the Lipschitz 
constant Lsjjj is known. 

2) Variable step-size: In scenarios where the knowledge of 
the system parameters, e.g. Lyu, is not available, one can use 
the diminishing step-size rule ( fTTI i. Note also that, under a 
diminishing step-size rule, convergence is guaranteed for any 
choice of r > C0and Hi(x)-c ffi 1^0. We will show in the 
next section that a diminishing step-size rule is also useful to 
allow an inexact computation of the best-response (x™ , r) 
while preserving convergence of the algorithm. Two classes 
of step-size rules satisfying (flTt are: given 7 = 1, 



Rule#l: 



Rule#2: 



7" 



7" 



7 "- 1 (l-e 7 "- 1 ) 



7" 



a(n) 



11 = 1, . 



1. 



(18) 
, (19) 



1 + /8(n) ' 

where in (fT8l e G (0, 1) is a given constant, whereas in ( fT9l 
a(n) and /3(n) are two nonnegative real functions of n > 1 
such that: i) < a{n) < /3(n); and ii) a(n)/(3(n) — > as 
n —7- 00 while ^ n (a(n)//3(n)) = 00. Examples of such a(n) 
and (3(n) are: a(n) =aor a(n) = log(n) a , and /3(n) ~ f3n 
or /3(n) = /3 ^/n, where a, /3 are given constants satisfying 
a G (0, 1), /3 G (0, 1), and a</3. 

Another issue to discuss is the choice of the free posi- 
tive definite matrices Hj(y). Mimicking (quasi-)Newton-like 
schemes |36l , a possible choice is to consider for Hj(x") a 
proper (diagonal) uniformly positive definite "approximation" 
of the Hessian matrix V x .[/(x"). The exact expression to 
consider depends on the amount of signaling and computa- 
tional complexity required to compute such a Hj(x"), and 
thus varies with the specific problem under consideration. 

3) On the choice of Ci's: In general, more than one 
(feasible) choice of {Ci} is possible for a given social function, 
resulting in different decomposition schemes. Some illustrative 
examples are discussed next. 

Example #1— (Proximal) gradient algorithms: If each Ci = 
and I = If, xc 4 (x n ,Ti) reduces to the gradient response 
(0 (possibly with a proximal regularization). It turns out 
that (exact and inexact) gradient algorithms along with their 
convergence conditions are special cases of our framework. 
Note that if Si = for every i (i.e., no convexity whatsoever 
is present in U), this is the only possible choice, and indeed 
our approach reduces to a form of gradient method. On the 
other hand, as soon as at least some Si 7^ 0, we may depart 
from the gradient method and exploit the available convexity. 
Example #2—Pricing algorithms in JT): Suppose that / = If, 
and each Si = {i} (implying that /j(»,x_i) is convex on JCi 
for any x_i G /C_j). By taking eachCi = {i} and Hj(x") = I, 
we obtain the pricing -based algorithms in [T]: 



Xi(x",7i) = argmin /iCx^x^J+TTiCx") 5 



Xi+- 



where 7Ti(x n ) = 2~2j^i ^xi/j(x n ). Algorithm Q] based on the 
above best-response implements naturally a pricing mecha- 
nism; indeed, each 7Tj(x™) represents a dynamic pricing that 

2 If 5Zjg£. /_) (•, x_j) is strongly convex on K.i for any x_i £ /C_;, we 
can also set r» = 0. 



measures somehow the marginal increase of the sum-utility of 
the other users due to a variation of the strategy of user i; 
roughly speaking, it works like a punishment imposed to each 
user for being too aggressive in choosing his own strategy 
and thus "hurting" the other users. Pricing algorithms based 
on heuristics have been proposed in a number of papers for 
the sum-rate maximization problem over SISO/SIMO/MIMO 
ICs I2TI - IF23I . ||3T1 . (37). However, on top of being sequential 
schemes, convergence of algorithms in the aforementioned 
papers is established under relatively strong assumptions (e.g., 
limited number of users, special classes of functions, specific 
channel models and transmission schemes, etc.), see l23l . 
The pricing in our framework is instead the natural conse- 
quence of the proposed SCA decomposition technique and 
leads to simultaneous algorithms that can be applied (with 
convergence guaranteed) to a very large class of problems, 
even when ED-ED, ED, ED fail. 

Example #3— (Proximal) Jacobi algorithms for a single jointly 
convex function: Suppose that the social function is a single 
(jointly) convex function /(xi,...,x/) on K, = Yli^i- Of 
course, this optimization problem can be interpreted as a 
special case of the framework ((T), with d = Si = {1} = If, 
for all i G I and /i(x) = /(x). Then, setting H^x") = I, 
the best-response (0 of each user i reduces to 



xc^x",^ 



argmin /(x 4 ,x^ 4 ) + - ||x, 



(20) 



Algorithm Q] based on (|20| | reads as a block-Jacobi schemes 
converging to the global minima of /(xi,...,X/) over JC 
(cf. Theorem [3). To the best of our knowledge, these are 
new algorithms in the literature; moreover their convergence 
conditions enlarge current ones; see, e.g., JU Sec. 3.2.4]). 
Quite interestingly, this new algorithm can be readily applied 
to solve the sum-rate maximization over MIMO multiple ac- 
cess channels (38], resulting in the first (inexact) simultaneous 
MIMO iterative waterfilling algorithm in the literature; we 
omit the details because of the space limitation. 
Example #4 —Algorithms for DC programming. The proposed 
framework applies naturally to sum-utility problems where the 
users' functions are the difference of two convex functions, 
namely: 



minimize 

X1,...,XJ 



subject to Xj G K, 



jf*(x)+£/f*(x) 



Vie I 



(21) 



where /? vx (x) and / l ccv (x) are convex and concave functions 
on JC, respectively. Letting 

/i(x)4£/~(x) and / 2 (x)^r(x), 

the optimization problem (f2TJ can be interpreted as a special 
case of the framework (|T), with If = {1, 2}, Ci = {1} for all 
i G I. The best-response (0 of each user i reduces then to 

x Ci (x'\t 4 ) - argminf /!(x) +7 ri (x") T x. 1 + § ||x, - x^ 2 } 

(22) 

where 7Tj(x n ) = V Xi /2(x") and Hi(x") = I. The above de- 
composition can be applied, e.g., to the sum-rate maximization 
(0, when all fi(x) = Wi x, with Wi > 0; see Sec. I VIII 
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B. Inexact Jacobi best-response schemes 

In many practical network settings, it can be useful to further 
reduce the computational complexity of the users' (convex) 
sub-problems (0 by allowing inexact computations of the 
best-response functions Xc 4 (x n ,r). Algorithm [2] below is a 
variant of Algorithm Q] in which suitable approximations of 
xc i (x™ , t) can be used. 

Algorithm 2: Inexact Jacobi SCA Algorithm 

Data : {e™} for i £ I, r > 0, {7™} > 0, x° e K. Set n = 0. 
( S . 1 ) : If x" satisfies a termination criterion: STOP; 
( S . 2 ) : For all i £ X, solve © within the accuracy ef : 

Find z? s.t. ||zj l - x Cl (x r \ r) || < e"; 
( S . 3 ) : Set x ,l+1 = x" + 7" (z" - x"); 
( S . 4 ) : n <- n + 1, and go to ( S . 1 ) . 

The error term e™ in Step 2 measures the accuracy used 
at iteration n in computing the solution xc,j (x",t) of each 
problem ©. Note that if we set e" = for all n and i, 
Algorithm |2] reduces to Algorithm Q] Obviously, the errors 
e"'s and the step-size 7 n 's must be chosen according to some 
suitable conditions, if one wants to guarantee convergence. 
These conditions are established in the following theorem. 

Theorem 4: Let {x"}™ =1 be the sequence generated by 
Algorithm |2] under the setting of Theorem [3] where however 
we reenforce assumption A4 by assuming that U is coercive 
on K,. Suppose that the sequences {7™} and {e™} satisfy the 
following conditions: i) 7™ e (0, 1]; ii) 7™ — > 0; iii) J2 n 7™ = 

+00; iv) J2 n (7 11 ) 2 < +°°; and v ) J2 n e i"y n < +°° for a11 

i = 1, . . . , I. Then, the conclusions of Theorem [3] holds. 

Proof: See Appendix B. ■ 

As expected, in the presence of errors, convergence of 
Algorithm [2] is guaranteed if the sequence of approximated 
problems (0 is solved with increasing accuracy. Note that, 
in addition to requiring e" — > 0, condition v) of Theorem 
|4] imposes also a constraint on the rate by which the e™ 
go to zero, which depends on the rate of decrease of {7"}. 
Two instances of step-size rules satisfying the summability 
condition iv) are given by (TT~8T > and (some choices of) ( fT9l >. 
An example of error sequence satisfying condition v) is 
e" < <H 7™, where c, is any finite positive constant. Interesting, 
such a condition can be forced in Algorithm |2] in a distributed 
way, using classical error bound results in convex analysis; 
see, e.g., El Ch. 6, Prop. 6.3.7]. 

Finally, it is worth observing that Algorithm |2] (and [TJ with 
a diminishing step-size rule satisfying i)-iv) of Theorem [4] is 
also robust against (stochastic) errors on the price estimates, 
due to an imperfect communication scenario (random link 
failures, noisy estimate, quantization, etc.). Because of the 
space limitation, we do not further elaborate on this here. 

C. (Inexact) Gauss-Seidel best-response schemes 

The Gauss-Seidel implementation of the proposed SCA 
decomposition is described in Algorithm [3] below, where the 
users solve sequentially, in an exact or inexact form, the 
convex subproblems ®. In the algorithm, we used the notation 



Algorithm 3: Inexact Gauss-Seidel SCA Algorithm 

Data : {e*} for i £ 1, r > 0, {7*} > 0, x° e K. Set t = 0. 
( S . 1 ) : If x* satisfies a termination criterion: STOP; 
(S . 2) : For i = 1, 

a) Find z* s.t. \\z\ ((x*+ 1 ,x*>),t) || < e\; 

b) Set x* +1 ^x*+7/ (zt-xf) 
( S . 3 ) : t <- t + 1, and go to ( S . 1 ) . 

Note that one round of Algorithm[3](i.e., t i— t+1) wherein 
all users sequentially update their own strategies, corresponds 
to I consecutive iterations n of the Jacobi updates described 
in Algorithms Q] and [2] In Appendix C we prove that, quite 
interestingly, Algorithm [3] can be interpreted as an inexact 
Jacobi scheme based on the best-response xc (•, t), satisfying 
Theorem |4] It turns out that convergence of Algorithm [3] 
follows readily from that of Algorithm |2] and is stated next. 

Theorem 5: Let {x"}™ =1 be the sequence generated by 
Algorithm [3] under the setting of Theorem |4] Then, the same 
conclusions of Theorem |4] hold. 

Proof: See Appendix C. ■ 

V. The Complex Case 

In this section we show how to extend our framework to 
sum-utility problems where the users' optimization variables 
are complex matrices. This will allow us to deal with the 
design of MIMO multiuser systems. Let us consider the 
following sum-utility optimization: 

minimize U(X) 4 £ fi (X) 
X: X tei t (23) 

subject to Xj € X i: Wi e I, 



where X = (Xi)igx, with Xi € 



being the (matrix) 



strategy of user i, X t C C n ' xm % and / 4 : X -> M, with 
X = Y\ iex Xc let define also X_ t = H jjti Xj. We study g3) 
under the same assumptions A1-A4 stated for the real case, 
where in A2 the differentiability condition is now replaced by 
the R-differentiability (see, e.g., J39), ED), and in A3 U(X) 
is required to have Lipschitz conjugate-gradient Vx*t^(X) 
on JC, with constant L^ Ut where X* is the conjugate of X. 

A motivating example. An instance of (l23l is the MIMO 
version of (0: 



maximize V] (Ri(Qi, Q-i)) 



(24) 



t+1 



and x*> = (x*,...,x* 7 ). 



subject to Qi £ Qi, Vi £ X. 

where i?j(Qj, Q-i) is the rate over the MIMO link i, 

Ri(Qi,Q-i) = logdet (i + H^RiCQ-iJ-^iQi) , (25) 

Qi is the covariance matrix of transmitter i, Rj(Q_i) 
R , + J2j^i Hy QjH^ is the covariance matrix of the mul- 
tiuser interference plus the thermal noise Ft Rj (assumed to be 
full-rank), with Q_i = (Qj)jjti, Hy is the channel matrix 
between the 7-th transmitter and the i-th receiver, and Qi is 
the set of constraints of user i, 

Q l = {Qi £ C niXni : Q, h 0, tr(Q0 < P t , Q t £ Z. t } . 

In Qi we also included an arbitrary convex and closed set Zi, 
which allows us to add additional constraints, such as: i) null 
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constraints XJfQi = 0, where U G C rii xn is a full rank matrix 
with r, < n,; ii) soft-shaping constraints tr (G^QjGj) <If ve ; 
iii) peak-power constraints A max (F^QiF^) < 7? eaIc ; and iv) 
per-antenna constraints [Qi]kk<otik- Note that the optimization 
problems in 11231 . 11241 . Il26l are special cases of (|24"1 >. 



A. Distributed decomposition algorithms 

At the basis of the proposed decomposition techniques 
for d23l there is the (second order) Taylor expansion of a 
continuously R-differentiable function / : C nxm -> R gO): 

/(X + AX) - /(X) a 2 (AX, V X ./(X)> 

+ i vec([AX, AX*]) tf Hxx./(X) vec([AX, AX*]), 

(26) 

where (A, B) = tr(A H B), vec(») denotes the "vec" operator, 
and "Hxx*/(X) is the so-called augmented Hessian of /, 
defined as gUI 

v f rx)A 9 f 9/(x) ^ 

nocx-M ; 5vec([X,X*]) T I ,9vec([X*,X]) T y ' 

(27) 

In l40l . we proved that % XX «/(X) plays the role of the 
Hessian matrix for functions of real variables. In particular, 
/ is strongly convex on C nxm if and only if there exists a 
c-fc > 0, the constant of strong convexity of /, such that 

vec([Y,Y*]) ff Hxx*/(X)vec([Y,Y*]) >c f c\\Y\\ 2 F , (28) 



for all X € 



and Y G 



where 



de- 



notes the Frobenius norm. When (128> holds, we say that 

■H XX »/(X) is augmented uniformly positive definite, and 

A . . 

write %xx*/(X) - Cjcl >z J40J. If / is only convex but 
not strongly convex, then Cfc in (|28l l is zero. 

Motivated by the Taylor expansion ( f26t , and using the same 
symbols <Sj and Cj to denote the complex counterparts of Si 
and Ci introduced for the real case [cf. (fj)], let us consider 
for each user i the following convex approximation of U(X.) 
at X" : denoting by AX, = X, - X™, 



/ Cl (X,;X") 



(n Ci (X»), AX, 



with 



— vec([AX„ AX*]) ff H,(X") vec ([AX,, AX*]) 

n Ci (x»)^ 2 V x*/.(X)| X=X „, 

jec-i 



(29) 



(30) 



where "Hi(X") is any given 2nm x 2nm matrix such that 
.4 

Hi(X) -CHflh 0, for all X G X and some c Hi > 0. Note 
that if Hi(X) = I, the quadratic term in ((29) reduces to the 
standard proximal regularization Tj ||X, — X"|| „. Then, the 
best-response matrix function of each user is 

Xc,(X b ,t< 



argmin/ Ci (X,;X^ 



(31) 



Decomposition algorithms for (123) are formally the same 
as those proposed in Sec. |IV] for (Q3 [namely Algorithms 
QJ3), where the real-valued best-response map xc(x",r) is 
replaced with the complex- valued counterpart Xc (X" , t) = 
(Xc 4 (X n ,Ti))^_ 1 . Convergence conditions read as in The- 
orems |3]|5] under the following natural changes: i) Lye/ 



becomes L'^, u ; ii) the condition H,;(x) — c# ; I ^ for all 

A 

x G K reads as Hi(X) - c Hi l h 0, for all X G X; and 
iii) in the constant c T in Theorem |3^) [cf. (fTTt l c^f; and 
c/ ; (z_,) are replaced with and c/ c , (Z_j), respectively, 
where c/ c . (Z_j) is the constant of strongly convexity of 

Y,jecjj(*i x -i) on ^ t if Y,jecjj(*> x -«) are convex but 
not strongly convex on Xi, then c/ c . (Z_j) = 0], 

VI. Extensions and related works 

The key idea in the proposed SCA schemes, e.g., (IBTt , is 
to convexify the nonconvex part of U via partial linearization 
of Eiec_» /j( X )' resulting in the term (II Ci (X n ), AX,). In 
the same spirit of ll27l . Il32l . Il33l . it is not difficult to show 
that one can generalize this idea and replace the linear term 
(IIc i (X™), AX,) in ( f29l with a nonlinear scalar function 
II Ci («;X ra ) : Xi 3 X, t-> n Ci (X,;X"). All the results 
presented so far are still valid provided that lie, (•; X") enjoys 
the following properties: for all X" G X, 

PI) lie (»;X n ) is M-continuously differentiable on Xf, 

P2) Vx.n Ci (X?;X") = V\ £C ( V x ./,-(X«); 

P3) n Ci (X,;X") is continuous in (X,;X™) G X, x X. 

Similar conditions can be written in the real case for the 
nonlinear function 7tc i (»;'x n ) ■ fCi 3 Xj >->• 7rc i (x,;x rl ) 
replacing the linear pricing 7r^.Xj. It is interesting to compare 
P1-P3 with conditions in (27], EH, (55). First of all, our 
conditions do not require that the approximation function is 
a global upper bound of the original sum-utility function, a 
constraint that remains elusive for sum-utility problems with 
no special structure. Second, even when the aforementioned 
constraint can be met, it is not always guaranteed that the 
resulting convex subproblems are decomposable across the 
users, implying that a centralized implementation might be 
required. Third, SCA algorithms ll2Tl . 11321 . Il33l . even when 
distributed, are generally sequential schemes (unless the sum- 
utility has a special structure). On the contrary, the algorithms 
proposed in this papers do not suffer from any of the above 
drawbacks, which enlarges substantially the class of (large 
scale) nonconvex problems solvable using our framework. 

VII. Applications and Numerical Results 

In this section, we customize the proposed decomposition 
framework to the SISO and MIMO sum-rate maximization 
problems introduced in (|3) and (|24) . respectively, and compare 
the resulting new algorithms with state-of-the-art schemes 
J23), ED, (|29), |30), J33]. Quite interestingly, our algorithms 
are shown to outperform current schemes, in terms of conver- 
gence speed and computational complexity, while reaching the 
same sum-rate. It is worth mentioning that this was not obvious 
at all, because algorithms in ll23l. l24l. Il29l. 11301. Il33l are ad- 
hoc schemes for the sum-rate problem, whereas our framework 
has been introduced for general sum-utility problems. 

A. Sum-Rate Maximization over SISO ICs 

Consider the social problem (O, with fa (x) = Wi x, where 
Wi are positive given weights; to avoid redundant constraints, 
let also assume w.l.o.g. that all the columns of W, are linearly 
independent. We describe next two alternative decompositions 
for © corresponding to differ choices of If and {Ci}. 



1) Decomposition #1— Pricing Algorithms: Since each 
user's rate Ti{pi, P-i) is concave in p.; G Pj, a natural choice 
is If = 2" and Cj = {i}, which leads to the following class of 
strongly concave subproblems [cf. d7}]: given p ra = (p™)f =1 
and choosing H;(p") = I, the best-response of user i is 

Pi(p n ) = 



2) Decomposition #2— DC Algorithms: An alternative class 
of algorithms for the sum-rate maximization problem under 
consideration can be obtained exploring the D.C. nature of 
the rate functions (cf. Example #4 in Sec. II V- Al l. The sum- 
rate can indeed be decomposed as the sum of a concave and 
convex function, namely U(p) = /i(p) + /2(p), where 



argmax jwi ^(p^p^) 
p t G Vi 



^(p") T p-iiiPi-p?n 2 } ' A(p) ± ^^io g (^ fc +^i^(fc)r 



Pjk) 
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where 7Tj(p") = (7r ife (p n ))f =1 is the pricing factor, given by / 2 (p) = - ^ w ?; log(a? 



fc T ^ 0)1 Pjfc) 



A/i denotes the set of neighbors of user i, i.e., the set of 
users j's which user i interferers with; and snr" fc and mui™ fe 
are the SINR and the multiuser interference-plus-noise power 
experienced by user j, generated by the power profile p": 



, |fftf(*)l a Pft 



snr jfe = 



mu± jk 

The best-response p;(p n ) can be computed in closed form 
(up to the multiplies associated with the inequality constraints 
in Vi) according to the following multi-level waterfilling-like 
expression l40l : 



P*AP 
1 



2 Pl 



(1 - (snr")- 1 ) + 



Ai" VlAi - n PF ° (1 + (snr»)-i)]^ + 4r^l 



(33) 



where o 

(1/snr^; 
vector 



denotes the Hadamard product, (snr") = 
k=1 and p, t = TTi(p n ) +'Wffj, i , with the multiplier 
chosen to satisfy the nonlinear complementarity 
condition (CC) < ^ _L If lax - W 4 p 4 (p" ) > 0. The optimal 
fi i satisfying the CC can be efficiently computed (in a finite 
number of steps) using a multiple nested bisection method 
as described in l40l Alg. 6]; we omit the details because of 
the space limitation. Note that, in the presence of the power 
budget constraint only (as in ll23l . l29l . Il30l ). /x i reduces to 
a scalar quantity \ii such that < /x, _L Pi — l T p;(p") > 0, 
whose solution can be obtained using the classical bisection 
algorithms (or the methods in PTI ). 

Given pj(p n ), one can now use any of the algorithms 
introduced in Sec. |IV] For instance, a good candidate is the 
exact Jacobi scheme with diminishing step-size (Algorithm 
[TJ, whose convergence is guaranteed if, e.g., rules in (TTFt or 
(fT~9T > are used for the sequence {7"} (Theorem O. Note that 
the proposed algorithm is fairly distributed. Indeed, given the 
interference generated by the other users [and thus the MUI 
coefficients mui™ fc ] and the current interference price 71^ (p n ), 
each user can efficiently and locally compute the optimal 
power allocation Pi(p n ) via the waterfilling-like expression 
(l33l >. The estimation of the prices 71-^. (p n ) requires however 
some signaling among nearby users. Interestingly, the pricing 
expression in (l32l as well as the signaling overhead necessary 
to compute it coincides with that in [23 1. But, because of 
their sequential nature, algorithms in 11231 require more CSI 
exchange in the network then our simultaneous schemes. 



which is an instance of OTT i with It = {1,2}. A natural 
choice of C; is then C; = {1} for all i el, resulting in the 
best-response: 

— / n \ A [ f / n \ / n\T 'i 11 ni|2l 

P4P )= argmax |/i(p,,p_J - m(p ) Pi - — ||pi - p 4 || |, 
Pi G Vi 



where Tr^p") ^ (7r lfc (p™))^ =1 , with 



\Hji (k) 



(34) 



ni, k (p n ) =-J2 w i 

We remark that the best-response Pj(p n ) can be efficiently 
computed by a fixed-point iterate, in the same spirit of [29 1 ; we 
omit the details because of the space limitation. Note that the 
communication overhead to compute the prices (l32t and (l34l 
is the same, but the computation of p;(p n ) requires more CSI 
exchange in the network than that of pi(p n ), since each user 
i also needs to estimate the cross-channels {\Hji (k)\ }jejvv 
Numerical Example. We compare now Algorithm 1 based on 
the best-response p;(p™) in d33l (termed SJBR), with those 
proposed in fl29l [termed SCALE and SCALE one-step, the 
latter being a simplified version of SCALE where instead of 
solving the fixed-point equation (16) in [291, only one iteration 
of (16) is performed], ED (termed MDP), and J30] (termed 
WMMSE). Since in the aforementioned papers only power 
budget constraints can be dealt with, to allow the comparison, 
we simplified the sum-rate maximization problem described 
above and considering only power budget constraints (and all 
Wi = 1). We assume the same power budget = P, noise 
variances af k = a 2 , and snr = P/a 2 = 3dB for all the users. 
We simulated SISO frequency channels with N = 64 subcarri- 
ers; the channels are generated as FIR filters of order L = 10, 
whose taps are i.i.d. Gaussian random variables with zero 
mean and variance l/(d^(L + l) 2 ), where dij is the distance 
between the transmitter j and the receiver i. All the algorithms 
are initialized by choosing the uniform power allocation, and 
are terminated when (the absolute value of) of the sum-utility 
error in two consecutive rounds becomes smaller than le-6. 
The accuracy in the bisection loops (required by all methods) 
is set to le-7. In our algorithm, we used the rule (TT¥t with 
e = le-2. In Fig. [U we plot the average number of iterations 
required by the aforementioned algorithms to converge (under 
the same termination criterion) versus the number of users; the 
average is taken over 100 independent channel realizations; 
we set dij j da = 3 and d^ = dji and da = djj for all i and 
j i. All the algorithms reach the same average sum-rate. The 
figure clearly shows that our SJBR outperforms all the others 
(note that SCALE and WMMSE are also simultaneous-based 
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Fig. 1: Average number of iterations versus number of users in SISO 
frequency-selective ICs. Note that all algorithms are simultaneous except 
MDP; this means that, at each iteration, in MDP there is only one user 
updating his strategy, whereas in the other algorithms all users do so). 

schemes). For instance, the gap with the WMMSE is about 
one order of magnitude for all the network sizes considered in 
the experiment. The good behavior of our scheme (requiring 
less iterations than other methods, with gaps ranging from 
few times to one order of magnitude) has been observed also 
for other choices of dij/du, termination tolerances, and step- 
size rules; we cannot present here more experiments because 
of space limitation; see [42] for the details. Note that SJBR, 
SCALE one-step, WMMSE, and MDP have similar per-user 
computational complexity, whereas SCALE is much more de- 
manding and is not appealing for a real-time implementation. 
Therefore, Fig. Q] provides also a roughly indication of the 
per-user cpu time of SJBR, SCALE one-step, and WMMSE. 

B. Sum-Rate Maximization over MIMO ICs 

Let us focus now on the MIMO formulation ( 124b . assuming 
fi (x) = Wi x, with Wi > 0. 

1) Decomposition #1: Pricing Algorithms: Choosing If 
I . d = {i}, and Hi(Q n ) = I, the best-response of user i is 

Qi(Q",Ti) = 

argmax {w t r t (Q t , Q",) - (II;(X"), Q<) - n ||Q. t - Q? f F ] 

Wlth rL(Q») 4 £ Wj ir^cr jTV 

where Mi is defined as in the SISO case, and 
T { , Q , ' = TUQ'V - (R ; ;Q'V H^Q?H?) . 

Note that, once the price matrix II; (Q n ) is given, the best- 
response Q;(Q",Ti) can be computed locally by each user 
solving a convex optimization problem. Moreover, for some 
specific structures of the feasible sets Qi, the case of full- 
column rank channel matrices H i; and T; = 0, a solution in 
closed form (up to the multipliers associated with the power 
budget constraints) is also available J24]. Given Qi(Q", r,), 
one can now use any of the algorithms introduced in Sec. 
Ivl To the best of our knowledge, our schemes are the first 
class of best-response Jacobi (inexact) algorithms for MIMO 
IC systems based on pricing with provable convergence. 

Complexity Analysis and Message Exchange. It is interesting 
to compare the computational complexity and signaling (i.e., 
message exchange) of our algorithms, e.g., Algorithm 1 based 
on the best-response Qj(Q",Tj) (termed MIMO-SJBR) with 



those of the schemes proposed in the literature for a similar 
problem, namely the MIMO-MDP m, El, and the MIMO- 
WMMSE |30l . For the purpose of complexity analysis, since 
all algorithms include a similar bisection step which generally 
takes few iterations, we will ignore this step in the computation 
of the complexity (as in J30l). Also, WMMSE and SJBR 
are simultaneous schemes, while MDP is sequential; we then 
compare the algorithms by givien the per-round complexity, 
where one round means one update of all users. Denoting by 
riT (resp. n R ) the number of antennas at each transmitter (resp. 
receiver), the computational complexity of the algorithms is: 

• MIMO-MDP: 0(l 2 (n T n 2 R + n\n R + n 3 R ) + In T ) 

• MIMO-WMMSE:0(/ 2 (ri T ?4 + n ^ nR + n 3 ) + I n 3 R )$M 

• MIMO-SJBR: 0(l 2 (n T n 2 R + n 2 T n R ) + I(n T + n 3 R )). 

It is clear that the complexity of the three algorithms is very 
similar, and equivalent in the case in which tit = n R {= n), 
given by (D(I 2 n 3 ). 

We remark that, in a real system, the MUI covariance 
matrices R;(Q_;) come from an estimation process. It is thus 
interesting to understand how the complexity changes when 
the computation of R;(Q_i) from R ni + 2~2j^ti TI .Q.Tlf' is 
not included in the analysis. We obtain the following: 

• MIMO-MDP: 0(l 2 {n T n 2 R + n%n R + n 3 R ) +In 3 T ) 

• MIMO- WMMSE: 0(l 2 (n 2 T n R + n 3 T ) + I(n 3 R + n T n 2 R )) 

• MIMO-SJBR: 0(l 2 (n T n 2 R + n 2 T n R ) + I(n T + n 3 R )). 

Finally, if one is interested in the time necessary to complete 
one iteration, it can be shown that it is proportional to the 
above complexity divided by /. 

As far as the communication overhead is concerned, the 
same remarks we made about the schemes described in the 
SISO setting, apply also here for the MIMO case. The only 
difference is that now the users need to exchange a (pricing) 
matrix rather than a vector, resulting in 0(I 2 n R ) amount of 
message exchange per-iteration for all the algorithms. 

2) Decomposition #2 -WMMSE Algorithms: In (301, the 
authors showed a very interesting result, namely: the MIMO 
problem d24l (under power constraints only) is equivalent to 
the following sum-MSE minimization: writing Qi = V^V^, 
V = (Vi)j =1 , and introducing the auxiliary matrix variables 
U4(U i )f =1 ,W4(W)f =1 , 

™ v / (W, U, V) 4 Y^w, (tr (Wi E.-CU, V))-logdet(W i )) 
s.t. tr(ViVf ) < F.'w, ^0, V* e I, 

(35) 

where Ej(U, V) is the MSE matrix at the receiver i (see (3) 
in IL301 ). The formulation (l35l l has some desirable properties, 
namely: i) / (W, U, V) is continuously (M-)differentiable 
with Lipschitz continuous (conjugate) gradient on the feasible 
set; ii) / (W, U, V) is convex in each variables W, U, V; 
iii) the minimization of / (W, U, V) w.r.t. to each W, U, V 
can be performed in parallel by the users; and iv) the optimal 
solutions of the individual minimizations are available in 
closed form, see ll30l for details. We will denote such optimal 
solutions as Wj(U,V), U<(U,V), and V,(U,W), for all 
i € X, where we made explicit the dependence on the variables 
that are kept fixed. In ll30l the authors proposed to use the 
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(Gauss-Seidel) block coordinate descent method to solve ( |35| >, 
resulting in the so-called MIMO-WMMSE algorithm. 

It is not difficult to see that the formulation d35T l can be 
casted into our framework, resulting in the following best- 
response mapping for each user i: Xj (W™, U", V") = 
(w,(U",V™),U 4 (U",V"),V 4 (U",W n )). We can then 
compute a stationary solution of d35l l and thus ( T24l using any 
of the Jacobi algorithms introduced in the previous sections 
based on Xj (W n , U ra , V ra ) (or its inexact computation). Note 
that the computational complexity as well as the communica- 
tion overhead of such algorithms are roughly the same of those 
of the MIMO-WMMSE f30t 

Numerical Example. In Tables I and II we compare the 
MIMO-SJBR, the MIMO-MDP O, El, and the MIMO- 
WMMSE |23l . 1241 . in terms of average number of iterations 
required to reach convergence, for different number of users, 
normalized distances d = dij/du (with dij = dji and 
da = djj for all i and j ^ i), and termination accuracy 
(namely: le-3 and le-6). All the transmitters/receivers are 
equipped with 4 antenna; in the step-size rule (TT~8T > we set 
e = le-5; we simulated uncorrected fading channel model, 
where the coefficients are Gaussian distributed with zero mean 
and variance l/df^; and we set R ni = cr 2 I for all i, and 
snr = P/a 2 = 3dB. All the algorithms reach the same 
average sum-rate. 

Given the results in Tables I and II, the following comments 
are in order. The proposed SJBR outperforms all the others 
schemes in terms of iterations, while having similar (or even 
better) computational complexity. Interestingly, the iteration 
gap with the other schemes reduces with the distance and the 
termination accuracy. More specifically: i) SJBR seems to be 
much faster than all the other schemes (about one order of 
magnitude) when d^/du = 3 [say low interference scenarios], 
and just a bit faster (or comparable to MIMO-WMMSE) when 
dij/du = 1 [say high interference scenarios]; and ii) SJBR is 
much faster than all the others, if an high termination accuracy 
is set (see Table I). Also, the convergence speed of SJBR is 
not affected too much by the number of users. Finally, in our 
experiments, we also observed that the performance of SJBR 
are not affected too much by the choice of the parameter e 
in the (1181) : a change of e of many orders of magnitude leads 
to a difference in the average number of iterations which is 
within 5%; we refer the reader to ll42ll for details, where one 
can also find a comparison of several other step-size rules. We 
must stress however that MIMO-MDP and MIMO-WMMSE 
do not need any tuning, which is an advantage with respect to 
our method. 

VIII. Conclusion 
In this paper, we proposed a novel decomposition frame- 
work, based on SCA, to compute stationary solutions of 
general nonconvex sum-utility problems (including social 
functions of complex variables). The main result is a new 
class of convergent distributed Jacobi (inexact) best-response 
algorithms, where all users simultaneously solve (inexactly) 
a suitably convexified version of the original social problem. 
Our framework contains as special cases many decomposition 
methods already proposed in the literature, such as gradient 



algorithms, and many block-coordinate descent schemes for 
convex functions. Finally, we tested our methodology on some 
sum-rate maximization problems over SISO/MIMO ICs; our 
experiments show that our algorithms are faster than ad- 
hoc state-of-the-art methods while having the same (user) 
computational complexity in the SISO case and similar (or 
better) complexity in the MIMO case. Some interesting future 
directions of this work are under investigation, e.g., how to 
adaptively choose the step-size rule (so that no a-priori tuning 
is needed), and how to generalized our framework to scenarios 
when only long-term channel statistics are available. 
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TABLE I: Average number of iterations (termination accuracy=le-6) 
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TABLE II: Average number of iterations (termination accuracy=le-3) 



Appendix 

For notational simplicity, in the following we will omit in 
each Xc 4 (y, Tj) [and x.c(y,T)] the dependence on Ci and tj, 
and write x;(y) [and x(y)]; also, we introduce fc t (Xj, x_j) = 
Ejec. fj( x ii x -i) and /c_»(xi,x_i) = Ejec-i fj(*i,x-i)- 

A. Proof of Proposition Q] 

Because of the space limitation we prove only (b)-(d); the 
proof of (a) is based on similar steps. We start proving (c). 
(c): Given y G JC, by definition, each Xj(y) is the unique 
solution of the problem (O and thus satisfies the minimum 
principle: for all z; G /Cj, 
(zj -£ !; (y)) T 

(V Xi / Ci (xi(y), y_i) + 7r Ci (y) + n Hi(y) (%(y) - y 4 )) > 0. 

(36) 

Summing and subtracting V Xi /c ; (yi, y-t) in (TSoT l. choosing 
Zj = and using 7r Ci (y) = V Xi /c_i(y), we get 

(yi - x»(y)) T (V x ,/c,(x,(y), y_i) - V Xl /c,(y l , y-t)) 
+ (y l -x l (y)) T V Xll 7(y) 
- n (Xi(y) - Yi ) T Hi(y) (%(y) - y t ) > 0, 

(37) 

for all i 6l Invoking the (strong) convexity of fd( 9 , Y-i) 
on JCi [and thus the (strong) monotonicity of V Xi /c j ( , ) y-i)L 
and using (|37| |. we obtain 

(Yi - x 8 (y)) T V Xl [/(y) > c T \\%(y) - y£ , (38) 

for all i G I. Summing d38l over i we obtain ( fl2] i. 
(b): Let x* G K, be a fixed point of x(y), that is x* = x(x*). 
By definition, each x;(y) satisfies (l36i l. for any given y G JC. 
Setting y = x* and using x* = x(x*), ( T36] l reduces to 

( Zi -x*) T V Xil 7(x*)>0 , (39) 

for all Zj G ICi and i G I. Taking into account the Cartesian 
structure of JC and summing (|39l over i G 1 we obtain 



11 



(z - x*) T V x ?7(x*) > 0, for all z G JC, with z = (z,)f =1 ; 
therefore x* is a stationary solution of (Q~). 

The converse holds because i) x(x*) is the unique optimal 
solution of (0 with y = x*, and ii) x* is also an optimal 
solution of (0, since it satisfies the minimum principle, 
(d): It follows readily from ([38} . □ 



< 



B. Proof of Theorems \3\ and [4] 

We prove Theorem |4) Theorem |3b) is a special case; the 
proof of simpler Theorem [2 a) is omitted and can be obtained 
following similar steps. The line of the proof is based on 
standard descent arguments, but suitably combined with the 
properties of x(y) (cf. Prop. [T), and the presence of errors 
{e"}. For any given n > 0, the Descent Lemma 061 yields 



J7(x" +1 ) < U (x") + 7™ V x [/ (x") T (z™ - x") 



(40) 



2 

with z" = (zf)f =1 , and zj 
gorithm [2). Using ||z" — x" 



(41) 



lim y^7* llx(x') - x'll 2 < +oo. 



(44) 



Since U(x) is coercive, U(x) > min ye ^ U(y) > — oo, 
implying that {{/ (x")}„ is convergent; it follows from (l44l 
and E"=o 7" = oo that lim inf .™ ||x(x") - x™|| = 0. 

Using Prop.Q] we show next that lirrin^oo ||x(x") — x ra || = 
0; for notational simplicity we will write Ax(x") = x(x") — 
x n . Suppose, by contradiction, that lim sup„_ > . C)0 || Ax(x™)j| > 
0. Then, there exists a 5 > such that ||Ax(x")j| > 28 for 
infinitely many n and also || Ax(x")|| < 8 for infinitely many 
n. Therefore, one can always find an infinite set of indexes, 
say Af, having the following properties: for any n € Af, there 
exists an integer i n > n such that 

||Ax(x™)|| < 5, ||Ax(x 4 ")|| > 28 (45) 
5 < ||Ax(x J ')|| < 28 n<j< i n . (46) 

Given the above bounds, the following holds: for all n G Af, 
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(a) 
< 



Ixfx 1 ") -x(x n )|| + ||x l " -x" 



(6) 

< 



< (1 +L) ||x l " -x"|| 

(l + ^^fdlA^) 



|z* -x(x')| 



t—n 



< 



(l + L) (2<5 + e max ) 



in-l 
t—n 



(47) 



(48) 



where (a) follows from (|45) and (|46) ; (b) is due to Prop. 
[TJa); (c) comes from the triangle inequality and the updating 
rule of the algorithm; and in (d) we used (|45) . (146) . and 



||z* — x(x*)| < X^£*, where e 11 
It follows from (|48j that 



defined in Step 2 (Al- 

| 2 < 2||x(x' 1 ) -x"|| 2 + 
2£ 4 ||s? -Xi(x n )f < 2||x(x") -x"j| 2 +2£ 4 (ef) 2 , where 
in the last inequality we used ||z" — Xj(x")|| < e", and 

VJJ (x") T (z" - x(x" ) + x(x" ) - x") < 

-Or ||x(x") - x"|| 2 + J2 t e? ||V Xl [/(x™)|| , 
which follows from Prop. Q~lc), (l40l yields: for all n > 0, 

(7(x" +1 ) < [/(x")- 7 "(c T -7"i V y)l|x(x")-x"|| 2 +T n , 

(42) 

where T n 4 7 « £. £ » ||V Xl f/(x")|| + ( 7 «) 2 L w E;^") 2 - 
Note that, under the assumptions of the theorem, E^Lo < 
oo. Since 7" — > 0, we have for some positive constant ft and 
sufficiently large n, say n>n, 

U (x" +1 ) < [/ (x") - 7 "ft l|x(x") - x"|| 2 + T n 

< U (x«) - ft EL fi 7* l|x(x*) - x*|| 2 + ElU 

It follows from (g3J and ^„ < 00 that either {U (x™)} -> 
—00 or else {U (x 11 )} converges to a finite value and 



liminf > 7* > — 



> 0. 



(49) 



(l + L)(2,5 + e - , 

We show next that d49l is in contradiction with the conver- 
gence of {J7(x™)}„. To do that, we preliminary prove that, 
for sufficiently large n E Af, it must be ||Ax(x")|| > 8/2. 
Proceeding as in (l48l i. we have: for any given n € Af, 

|| Ax(x" +1 )|| - || Ax(x™)|| < (1 + L) ||x ,l+1 - x™|| 

< (1 + L)Y l (||Ax(x")|| +e max ) . 

It turns out that for sufficiently large n G Af so that (1 + 

L)7" < 5/(8 + 2e max ), it must be 

||Ax(x")|| > 8/2; (50) 

otherwise the condition || Ax(x" +1 )|| > 8 would be violated 
[cf. (ffoTll. Hereafter we assume w.l.o.g. that (l50t holds for all 
n € Af (in fact, one can alway restrict {x. n } n( zj^ to a proper 
subsequence). 

We can show now that (l49l is in contradiction with the 
convergence of {£/(x")}„. Using (03) (possibly over a sub- 
sequence), we have: for sufficiently large n € A/", 



J7 



< 



[/(x")-/3 2 ^7*||Ax(x*)| 



|Ax(x'")|| - ||Ax(x") 



< [/(x")-/3 2 ( ( 5 2 /4)^7*+^T t (51) 

i— n t—n 

where in (a) we used (06) and (150) . and ft is some positive 
constant. Since {[/(x n )} n converges and Yl^Lo T„ < 00, (IBT) 
implies limjv"9n->oo Et^Ln 1 7* = ^> which contradicts ((49). 

Finally, since the sequence {x ra } is bounded [due to the 
coercivity of C/(x) and the convergence of {J7(x ra )}„], it has 
at least one limit point x that must belong to K,. By the con- 
tinuity of x(») [Prop. [Ha)] and liirin^oo ||x(x") — x" || = 0, 
it must be x(x) = x. By Prop. |TJb) x is also a stationary 
solution of the social problem (H). □ 

C. Proof of Theorem \5\ 

The main idea of the proof is to interpret Algorithm |3] as an 
instance of the inexact Jacobi scheme described in Algorithm 
12 and show that Theorem |4] is satisfied. It is not difficult to 
show that this reduces to prove that, for all i = 1, . . . , I, the 
sequence z* in Step 2a) of Algorithm |3] satisfies 

\\^\-%{^)\\<el (52) 
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for some {e*} such that J^t *H 7* < 00 ■ The following holds 
for the LHS of <|52): 



^-Xi(x*)||<||x j (x*+ 1 lX ,y-£ j (x t )| 
fM ^(^x^-x^x*) 



(<-) 



4: 



(d) 



where (a) follows from the error bound in Step 2a) of 
Algorithm [3j in (b) we used Prop. QJ); (c) follows from Step 
2b); and in (d) we used Prop. QJ), with fa < oo being a 
positive constant. It turns out that ( 1521 is satisfied choosing 

□ 



et^L 1 t p i + L 1 t Y. 3<l e t 3 +ef 



References 

[1] G. Scutari, D. P. Palomar, F. Facchinei, and J.-S. Pang, "Distributed 
dynamic pricing for MIMO interfering multiuser systems: A unified 
approach," in Proc. of Int. Conf. on NETwork Games, COntrol and 
Optimization (NetGCooP 2011), Paris, France, Oct. 2011, pp. 12-14. 

[2] D. P. Palomar and M. Chiang, "Alternative distributed algorithms for 
network utility maximization: Framework and applications," IEEE Trans, 
on Automatic Control, vol. 52, no. 12, pp. 2254-2269, Dec. 2007. 

[3] M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, "Layering 
as optimization decomposition: A mathematical theory of network 
architectures," Proc. of the IEEE, vol. 95, no. 1, pp. 255-312, Jan. 2007. 

[4] D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computa- 
tion: Numerical Methods, 2nd ed. Athena Scientific Press, 1989. 

[5] Z.-Q. Luo and S. Zhang, "Spectrum management: Complexity and 
duality," IEEE J. Set Topics Signal Process., vol. 2, no. 1, pp. 57-72, 
Feb. 2008. 

[6] M. Hong and Z.-Q. Luo, "Signal processing and optimal resource 
allocation for the interference channel," Elsevier e-Reference-Signal 
Processing, 2013. Available at http://arxiv.org/pdf/1206.5144vl.pdf. 
[7] W. Yu, G. Ginis, and J. M. Cioffi, "Distributed multiuser power control 
for digital subscriber lines," IEEE J. Sel. Areas Commun., vol. 20, no. 5, 
pp. 1105-1115, June 2002. 
[8] Z.-Q. Luo and J.-S. Pang, "Analysis of iterative waterfilling algorithm 
for multiuser power control in digital subscriber lines," EURASIP Jour, 
on Applied Signal Processing, vol. 2006, pp. 1-10, May 2006. 
[9] G. Scutari, D. P. Palomar, and S. Barbarossa, "Optimal linear precoding 
strategies for wideband noncooperative systems based on game theory — 
part I&JJ: Nash equilibria and distributed algorithms," IEEE Trans. 
Signal Process., vol. 56, no. 3, pp. 1230-1267, March 2008. 

[10] , "Asynchronous iterative water-filling for Gaussian frequency- 
selective interference channels," IEEE Trans, on Information Theory, 
vol. 54, no. 7, pp. 2868-2878, July 2008. 

[11] R. Cendrillon, J. Huang, M. Chiang, and M. Moonen, "Autonomous 
spectrum balancing for digital subscriber lines," IEEE Trans. Signal 
Process., vol. 55, no. 8, pp. 4241-4257, Aug. 2007. 

[12] G. Scutari, D. P. Palomar, and S. Barbarossa, "Competitive design of 
multiuser MIMO systems based on game theory: A unified view," IEEE 
J. Sel. Areas Commun., vol. 26, no. 7, pp. 1089-1103, Sept. 2008. 

[13] , "The MIMO iterative waterfilling algorithm," IEEE Trans. Signal 

Process., vol. 57, no. 5, May 2009. 

[14] G. Scutari and D. P. Palomar, "MIMO cognitive radio: A game theoreti- 
cal approach," IEEE Trans. Signal Process., vol. 58, no. 2, pp. 761-780, 
Feb. 2010. 

[15] E. Larsson, E. Jorswieck, J. Lindblom, and R. Mochaourab, "Game 
theory and the flat-fading gaussian interference channel," IEEE Signal 
Process. Mag., vol. 26, no. 5, pp. 18-27, Sept. 2009. 

[16] A. Leshem and E. Zehavi, "Game theory and the frequency selective 
interference channel," IEEE Signal Process. Mag., vol. 26, no. 5, pp. 
28-40, Sept. 2009. 

[17] F. Facchinei and J.-S. Pang, Finite-Dimensional Variational Inequalities 
and Complementarity Problem. Springer- Verlag, New York, 2003. 

[18] J.-S. Pang, G. Scutari, D. P. Palomar, and F. Facchinei, "Design of 
cognitive radio systems under temperature-interference constraints: A 
variational inequality approach," IEEE Trans. Signal Process., vol. 58, 
no. 6, pp. 3251-3271, June 2010. 



[19] G. Scutari, D. Palomar, F. Facchinei, and J.-S. Pang, "Flexible design 
of cognitive radio wireless systems: From game theory to variational 
II inequality theory," IEEE Signal Process. Mag., vol. 26, no. 5, pp. 107— 
'II 123, Sept. 2009. 

[20] , "Convex optimization, game theory, and variational inequality 

theory in multiuser communication systems," IEEE Signal Process. 
Mag., vol. 27, no. 3, pp. 35^19, May 2010. 

[21] J. Huang, R. Berry, and M. L. Honig, "Distributed interference com- 
pensation for wireless networks," IEEE J. Sel. Areas Commun., vol. 24, 
no. 5, pp. 1074-1084, May 2006. 

[22] F. Wang, M. Krunz, and S. Cui, "Price-based spectrum management in 
cognitive radio networks," IEEE J. Sel. Topics Signal Process., vol. 2, 
no. 1, pp. 74-87, Feb. 2008. 

[23] D. Schmidt, C. Shi, R. Berry, M. Honig, and W. Utschick, "Distributed 
resource allocation schemes: Pricing algorithms for power control and 
beamformer design in interference networks," IEEE Signal Process. 
Mag., vol. 26, no. 5, pp. 53-63, Sept. 2009. 

[24] S.-J. Kim and G. B. Giannakis, "Optimal resource allocation for MIMO 
ad hoc cognitive radio networks," IEEE Trans, on Information Theory, 
vol. 57, no. 5, pp. 3117-3131, May 2011. 

[25] L. Grippo and M. Sciandrone, "On the convergence of the block 
nonlinear gauss-seidel method under convex constraints," Operations 
Reseach Letters, vol. 26, no. 3, pp. 127-136, April 2000. 

[26] S. Ye and R. S. Blum, "Optimized signaling for MIMO interference 
systems with feedback," IEEE Trans. Signal Process., vol. 51, no. 11, 
pp. 2839-2848, Nov. 2003. 

[27] M. Chiang, C. W. Tan, D. P. Palomar, D. O. Neill, and D. Julian, "Power 
control by geometric programming," IEEE Trans. Wireless Commun., 
vol. 6, no. 7, pp. 2640-2651, July 2007. 

[28] P. Tsiaflakis, M. Diehl, and M. Moonen, "Distributed spectrum man- 
agement algorithms for multiuser dsl networks," IEEE Trans. Signal 
Process., vol. 56, no. 10, pp. 4825^1843, Oct. 2008. 

[29] J. Papandriopoulos and J. S. Evans, "Scale: a low-complexity distributed 
protocol for spectrum balancing in multiuser dsl networks," IEEE Trans, 
on Information Theory, vol. 55, no. 8, pp. 3711-3724, Aug. 2009. 

[30] Q. Shi, M. Razaviyayn, Z.-Q. Luo, and C. He, "An iteratively weighted 
MMSE approach to distributed sum-utility maximization for a MIMO 
interfering broadcast channel," IEEE Trans. Signal Process., vol. 59, 
no. 9, pp. 4331^1340, Sept. 2011. 

[31] C. Shi, D. A. Schmidt, R. A. Berry, M. L. Honig, and W. Utschick, 
"Distributed interference pricing for the MIMO interference channel," 
in Int. Conf. on Comm. (ICC), Princeton, NJ, USA, June 14-18 2009. 

[32] B. R. Marks and G. P. Wright, "A general inner approximation algorithm 
for nonconvex mathematical programs," Operations Research, vol. 26, 
no. 2, pp. 681-683, 1978. 

[33] M. Razaviyayn, M. Hong, and Z.-Q. Luo, "A unified convergence 
analysis of block successive minimization methods for nonsmooth 
optimization," Arxiv.org, Oct. 2012. http://arxiv.org/abs/1209.2385vl. 

[34] M. Hong, Q. Li, Y.-F. Liu, and Z.-Q. Luo, "Decomposition by succes- 
sive convex approximation: A unifying approach for linear transceiver 
design in interfering heterogenous networks," Arxiv.org, Oct. 2012. 
http://arxiv.Org/abs/l 2 10. 1 507v 1 . 

[35] R. Cendrillon, W. Yu, M. Moonen, J. Verlinden, and T. Bostoen, 
"Optimal multiuser spectrum balancing for digital subscriber lines," 
IEEE Trans. Signal Process., vol. 54, no. 5, pp. 922-933, May 2006. 

[36] D. Bertsekas, Nonlinear Programming. Belmont, MA, USA: Athena 
Scientific, 2th Ed., 1999. 

[37] C. Shi, R. Berry, and M. Honig, "Distributed interference pricing for 
ofdm wireless networks with non-separable utilities," in 42nd Annu. 
Conf. Information Sciences and Systems, Princeton, NJ, USA, Mar. 2008. 

[38] W. Yu, W. Rhee, S. Boyd, and J. Cioffi, "Iterative water-filling for 
gaussian vector multiple access channels," IEEE Trans, on Information 
Theory, vol. 50, no. 1, pp. 145-151, Jan. 2004. 

[39] A. Hjorangnes, Complex-Valued Matrix Derivatives With Applications 
in Signal Processing and Communications. London: Cambridge 
University Press, May 2011. 

[40] G. Scutari, F. Facchieni, J.-S. Pang, and D. P. Palomar, "Real 
and complex monotone communication games," IEEE Trans, on 
Information Theory, (submitted, Nov. 2012). [Online]. Available: 
|htq37/arxiv.o rg/abs/i 2 1 2.6235 

[41] D. P. Palomar and J. Fonollosa, "Practical algorithms for a family of 
waterfilling solutions," IEEE Trans. Signal Process., vol. 53, no. 2, pp. 
686-695, Feb. 2005. 

[42] P. Song, "Numerical comparison of decomposition algorithms for 
nonconvex sum-utility problems," Dept. of Elect. Eng., Univ. at Buffalo, 
the State Univ. of New York, Tech. Rep., Jan. 2013. [Online]. Available: 
http://www.eng. buffalo. edu/$\sim$peiranso/SongTechRepNumCompJan 13.pdf. 



